Skip to main content

ArchiveBot: The Archive Team Crowdsourced Crawler

ArchiveBot is an IRC bot designed to automate the archival of smaller websites (e.g. up to a few hundred thousand URLs). You give it a URL to start at, and it grabs all content under that URL, records it in a WARC, and then uploads that WARC to ArchiveTeam servers for eventual injection into the Internet Archive (or other archive sites).

To use ArchiveBot, drop by #archivebot on EFNet. To interact with ArchiveBot, you issue commands by typing it into the channel. Note you will need channel operator permissions in order to issue archiving jobs. The dashboard shows the sites being downloaded currently.

There is a dashboard running for the archivebot process at http://www.archivebot.com.

ArchiveBot's source code can be found at https://github.com/ArchiveTeam/ArchiveBot.

14,503
RESULTS
rss


PART OF
Archive Team
Media Type
14,503
web
Year
9,211
2017
3,909
2016
942
2015
424
2014
16
2013
Topics & Subjects
12,377
archivebot
1
184.180.244.41
1
3dblogger.typepad.com
1
ahkscript.org
1
approachingaro.org
1
arstechnica.com
More right-solid
Collection
More right-solid
Creator
2,122
archive team
SHOW DETAILS
up-solid down-solid
eye
Title
Date Archived
Creator
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 1.5M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 1.3M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 922,666
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 815,047
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 560,291
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 513,655
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 455,988
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 445,259
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 404,365
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 401,676
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 400,000
favorite 1
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 387,824
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 386,742
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 376,118
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 375,262
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 350,999
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 334,323
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 332,950
favorite 1
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 328,435
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 316,041
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 312,440
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 311,839
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 309,921
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 308,473
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 303,576
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 303,066
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 299,157
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 298,538
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 297,492
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 293,175
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 291,851
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 290,284
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 286,252
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 286,034
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 280,385
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 279,882
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 279,791
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 273,801
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 273,175
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 267,323
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 266,695
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 265,512
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 264,922
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 263,673
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 262,634
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 259,057
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 258,633
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 257,877
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 257,132
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 255,936
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 254,819
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 254,455
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 253,698
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 253,567
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 252,532
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 252,159
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 251,854
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 251,853
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 249,474
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 248,464
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 245,116
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 243,058
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 241,794
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 241,439
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 239,772
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 238,851
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 237,296
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 234,562
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 233,043
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 232,556
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 232,357
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 231,515
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.